ABSTRACT
AI THEME ISSUE: How can health care organizations ensure that there is accountability of algorithms for accuracy, bias, and the wide range of unintended consequences when deployed in real-world settings? A machine-learning system for Covid-19 contact tracing serves as a model to scope out, develop, interrogate, and assess an algorithmic solution that produces improvements in care, mitigates risk, and enables evaluation by many stakeholders.
ABSTRACT
Anonymized smartphone-based mobility data has been widely adopted in devising and evaluating COVID-19 response strategies such as the targeting of public health resources. Yet little attention has been paid to measurement validity and demographic bias, due in part to the lack of documentation about which users are represented as well as the challenge of obtaining ground truth data on unique visits and demographics. We illustrate how linking large-scale administrative data can enable auditing mobility data for bias in the absence of demographic information and ground truth labels. More precisely, we show that linking voter roll data - -containing individual-level voter turnout for specific voting locations along with race and age - -can facilitate the construction of rigorous bias and reliability tests. Using data from North Carolina's 2018 general election, these tests illuminate a sampling bias that is particularly noteworthy in the pandemic context: older and non-white voters are less likely to be captured by mobility data. We show that allocating public health resources based on such mobility data could disproportionately harm high-risk elderly and minority groups. © 2021 Owner/Author.